Camera sensor data or images can be used as input to build a 3D map of an environment. In an augmented reality (AR) system, virtual objects can be inserted into a device display, the virtual objects interacting with or replacing real/physical world objects. An AR system can include input from a camera sensor to record real world objects as images or video, and display an AR view on the device display.
AR systems can implement a tracking system to estimate a user's position and orientation (pose) relative to a reference. Traditional tracking systems are limited by the type of reference tracked as well as by initialization procedures used to begin tracking. For example, in traditional tracking systems, a previously acquired reference or an artificial marker may be used to initialize tracking of a 3-Dimensional (3D) model of an environment. Otherwise, without the previously known reference, objects may appear at the wrong location or float around the map of the environment.
A tracking system utilizing a single camera may also rely upon initializing a 3D map from two subsequent reference images captured by the single camera. Creating a 3D map using traditional techniques based on the two reference images is only possible if the camera motion between the two reference images is appropriate, and also maintains enough overlap between the scenes in both images.
Traditional tracking implementations may also rely on direct user input to select the two reference images or may provide an additional visual target to record 6 degrees of freedom (6 DoF) camera motion before a 3D map can be initialized.
For example, some tracking methods require the user to perform a specific unintuitive motion sequence without visual feedback so that 3D reconstruction methods can be used to find a real plane of the target.
As a result of the above limitations of traditional methods of tracking technologies the current augmented reality user experience can feel unnatural. Moreover, most users are unlikely to know or understand the motions necessary for traditional tracking initialization using two reference images. Typical users are also frequently confused as to why they should have to perform the specific motions before an augmented reality system can display tracking updates for a scene.
Accordingly, improved user-friendly 3D environment initialization and tracking are desired.